Crate ion_binary_rs

Ion Binary parser/encoder & Ion Hash in Rust

Ion binary is a production-ready library written in safe rust for parsing, encoding and hashing Amazon’s Ion binary format.

It parses, encodes and hashes anything you throw at it. Any failure to do so is a bug that we will fix and we will be very happy if you report them.

The code is mature and way more tested than Amazon’s alternatives, including their js library. Amazon, when implementing the “good tests” only check that it parses. We check that the value is the correct one too. Additionally, we seem to be implementing way more of the ion protocol than Amazon’s libraries, as we don’t have huge skip lists in the tests files. We test all their test suite, not only a few test.

How to use the library

First of all, you need to be aware of the trade offs that we took for this library:

• The API returns strings instead of Symbols. If needed we can add symbol, but we think string is the the most ergonomic way.
• When parsing/decoding you can add shared tables for binary blobs that doesn’t have all the required symbols.

We have implemented the whole amazon ion test-suite for parsing. Encoding and Hashing fully tested. We are working in expading the coverage. We would appreciate any bug you can report. You can check all the test for examples.

Example

Parsing

use ion_binary_rs::IonParser;

// This is the response from Amazon's QLDB introduction example using Rusoto
let ion_test = b"\xe0\x01\0\xea\xee\xa6\x81\x83\xde\xa2\x87\xbe\x9f\x83VIN\x84Type\x84Year\x84Make\x85Model\x85Color\xde\xb9\x8a\x8e\x911C4RJFAG0FC625797\x8b\x85Sedan\x8c\"\x07\xe3\x8d\x88Mercedes\x8e\x87CLK 350\x8f\x85White";

let mut parser = IonParser::new(&ion_test[..]);

println!("Decoded Ion: {:?}", parser.consume_all().unwrap())
// Decoded Ion: [Struct({"Color": String("White"), "Year": Integer(2019), "VIN": String("1C4RJFAG0FC625797"), "Make": String("Mercedes"), "Model": String("CLK 350"), "Type": String("Sedan")})]

Encoding

use ion_binary_rs::{IonEncoder, IonParser, IonValue};
use std::collections::HashMap;

let mut ion_struct = HashMap::new();

ion_struct.insert("Model".to_string(), IonValue::String("CLK 350".to_string()));
ion_struct.insert("Type".to_string(), IonValue::String("Sedan".to_string()));
ion_struct.insert("Color".to_string(), IonValue::String("White".to_string()));
ion_struct.insert(
    "VIN".to_string(),
    IonValue::String("1C4RJFAG0FC625797".to_string()),
);
ion_struct.insert("Make".to_string(), IonValue::String("Mercedes".to_string()));
ion_struct.insert("Year".to_string(), IonValue::Integer(2019));

let ion_value = IonValue::Struct(ion_struct);

let mut encoder = IonEncoder::new();

encoder.add(ion_value.clone());
let bytes = encoder.encode();

let resulting_ion_value = IonParser::new(&bytes[..]).consume_value().unwrap().0;

assert_eq!(ion_value, resulting_ion_value);

Hashing

use sha2::Sha256;
use ion_binary_rs::{IonHash, IonValue};
use std::collections::HashMap;

let mut ion_struct = HashMap::new();

ion_struct.insert("Model".to_string(), IonValue::String("CLK 350".to_string()));
ion_struct.insert("Type".to_string(), IonValue::String("Sedan".to_string()));
ion_struct.insert("Color".to_string(), IonValue::String("White".to_string()));
ion_struct.insert(
    "VIN".to_string(),
    IonValue::String("1C4RJFAG0FC625797".to_string()),
);
ion_struct.insert("Make".to_string(), IonValue::String("Mercedes".to_string()));
ion_struct.insert("Year".to_string(), IonValue::Integer(2019));

let ion_value = IonValue::Struct(ion_struct);

let hash = IonHash::digest::<Sha256>(&ion_value);

println!("{:X?}", hash);

Safety

In order to speed up the encoding of data, we use Uninit vector buffers, as otherwise it would require for the buffer to be set to zeroes and then to the correct values.

Contributing

We would be thrilled if you decide to check the library and/or contribute to it! Just open an issue or pull request and we can check what you would like to implement. Bug hunting and proposals are always welcomed. And of course, feel free to ask anything.

License

^{Licensed under either of Apache License, Version 2.0 or MIT license at your option.}
_{Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.}

Structs

• IonEncoder
  Allows to binary encode one or multiple IonValue.
• IonHash
  Ion Hash implementation. Once the hasher is initialized you can add new values to it and it will perform the dot operation internally. Once you added everything you want to add just call get() and it will provide you with a &u8 slice containing the hash.
• IonParser
  In order to use it call the new method and then the “consume_all” method.

Enums

• IonExtractionError
• IonParserError
  Indicated a problem in the binary blob internal structure. When all data is read the library will return IonParserError::BinaryError(ParsingError::NoDataToRead).
• IonValue
  The structure wrapping all possible return ion values by the IonParser.
• NullIonValue
  Instead of wrapping each IonValue in an Option in order to represent the null value, we opted to join all Null values in the IonValue::Null(_) which contains this struct. Here you can check what kind of null you got. We do this because we believe is more ergonomic and simplifies the API handling.
• ParsingError
  This errors indicate a problem in a primitive parsing. It comes always wrapped by the “BinaryError” of the error type “IonParserError”.
• SerdeJsonParseError
• Symbol
  A table symbol. It can b used together with the “with_shared_table” method in order to define new shared tables.
• SymbolContextError
  Errors that can happen related with the Symbol Table.